Recently, a correction for camera shake on an image shot by a shooting device such as a digital camera etc. is performed by combining continuously shot images with short exposure. The images are combined by continuously shooting a specific subject and aligning continuously shot images. The last composite image is obtained by combining (adding and averaging pixel values) a plurality of images of pixels at the same positions, but the number of images used in combining the images can be different depending on the areas.
Since there can be a movement of a subject in addition to camera shake in the continuously shot images, a subject can be multiplexed in a simple addition. If the correlation between the continuously shot images is detected, and there is no pixel difference, then the images are combined, and if there is a pixel difference, then a different image is not used in the combination, thereby preventing a multiplexed image (detection of motion area). If there is a displacement between the images, there are an area where an image overlays on another image and an area where images do not overlay on other images. A plurality of images can be combined in a common area, but cannot be combined in other areas. Thus, continuously shot images can be combined as a composite image in which a one image area, two combined image area, a three combined image area, etc. coexist. Noise of an image can be different the number of images used in the combination. For example, noise is reduced in a larger number of combined images because random noise is averaged by combining a plurality of images. Therefore, one image includes an area where there is a large amount of noise and an area where there is a small amount of noise.
Thus, to make a clear shot image, a sharpening process has been performed on such a composite image. However, in the sharpening process, the sharpness of an image can be improved in the sharpening process, but noise is emphasized. Therefore, a highest possible adjustment to improve the sharpness without emphasizing noise is normally made.
In addition, in an image as a result of a correction for camera shake on simply combined images, areas including different numbers of overlaid images coexist. Therefore, the amount of noise depends on the number of overlaid images, and when the sharpening process is performed on the basis of the area having a smaller number of overlaid images, the sharpness is reduced without a strong sharpening process. Additionally, the noise is emphasized if the process is performed on the basis of an area having a larger number of overlaid images.
Under the circumstances, the patent document 1 proposes the technology of decreasing or increasing the number of combined images depending on the motion vector (a motion area and a no-motion area) between the frames of the moving pictures, and inserting into the first frame of the moving pictures and combining the pixels of a plurality of frames depending on the number of combined images.
However, according to the patent document 1, it is difficult to convert a plurality of low resolution images into high resolution images in high image quality, the n-th frame and the (n+a)-th frame are input, and images are combined using a+1 images and an intermediate frame generated by the insertion from a+1 images.
On the other hand, images are images are combined from a plurality of images, two frames used in combining images depending on the motion vector are changed, and the interval a between the two frames is shortened. It is a “determination of an image used in the combination”, and no description is made as to what process is performed on an image as a combination result.
The patent document 2 proposes switching a method of combining images (image processing) depending on the result obtained from the motion vector for each area after combining images for each area.
However, according to the patent document 2, the shake of an image includes “subject shake” and “camera shake”, and the shake cannot be simultaneously removed in the same shake removing process. In addition, the camera shake and the subject shake are detected, and image processing is performed depending on the type of shake. Although the image processing is switched depending on the type of detected shame, the determination reference is the “type of shake”, not the number of combined images. The image processing is switched by a different trigger.
That is, the patent document 1 determines the frame used in combining images depending on the motion vector. The patent document 2 switches the processes depending on the type of shake.
The patent document 3 proposes a system of discriminating from a character area and a gray-scale image area using a reader for reading an original image, and processing (shading, outlining, etc.) each area using a different parameter.
However, according to the patent document 3, one image is divided into a plurality of areas and the areas are processed using different parameters, but it is an image separating method for dividing an image into a plurality of areas on the basis of the image combining process for a plurality of images and performing processes using different parameters, but not separating an area from one image.
The patent document 4 proposes a system of reading a shot image, separating a main area from a background area, and performing image processing on the main area or the background area by a color reproduction parameter.
However, according to the patent document 4, an image is separated and different image processing is performed on the image, but a main portion is separated from a background on the basis of one image. That is, a different image separating method is used.
The patent document 5 proposes a system of extracting only a subject from an image including a subject and an image not including a subject.
However, according to the patent document 5, an area is separated using a plurality of images, but the separating method is used to combine a plurality of images. Additionally, to perform different processes on the respective areas for a composite image, different processes are performed on the original image before the separation to easily perform an extracting operation.
The patent document 6 proposes image processing performed depending on an area by extracting a specific area from a color image.
However, if the number of combined images can be different depends on the image area, the amount of noise of a finally obtained image depends on the number of combined images. Therefore, the amount is not constant, and a noise removing process is to be performed. In addition, although a noise removing process can be performed only on a target portion by performing an image analysis (detecting an edge portion), the image analysis is accompanied by a complicated process, thereby incurring a prolonged processing time and a larger circuit. Additionally the image analysis does not always produce a desired result.
For example, when a noise removing process is performed, the process has been performed using the same parameter on the entire image, the noise removing process is performed on a portion including much noise but not performed on an edge portion, thereby performing the image area and switching parameters depending on the analysis result.
In the noise removing process, a filtering process is performed on an image. When it is performed on a noise area, noise is reduced. However, when it is performed on an edge portion, the edge becomes unclear, and the sharpness is degraded. Furthermore, if it is performed uniformly on the entire image using the same parameter, then there can be an area (an edge portion etc.) subject to a side effect. Therefore, the process can be moderately performed not to cause a side effect.    Patent Document 1: Japanese Laid-open Patent Publication No. 2000-194845    Patent Document 2: Japanese Laid-open Patent Publication No. 2003-134385    Patent Document 3: Japanese Patent No. 3023374    Patent Document 4: Japanese Patent No. 3408770    Patent Document 5: Japanese Laid-open Patent Publication No. 8-194113    Patent Document 6: Japanese Laid-open Patent Publication No. 11-266358